Unusual activity detection for monitoring applications
Abstract
Unusual activity detection is the process of identifying and detecting the activities which are different from actual or well-defined set of activities and attract human attention. We have developed an application to automatically detect unusual activities. Generally, humans follow a sequence in their actions. Based upon these sequences of activities, we will classify them into the usual and unusual activities.
A lot of human resources are required to monitor the vast number of surveillance cameras installed in public and private places. So, it is highly recommended to have an automatic algorithm which can detect unusual activities and trigger action.
Methodology
Our work on unusual activity detection is based on four steps leading to detection of unusual activity or anomaly in everyday activities of people. The four detection steps are,
1. Human detection using deep CNN object detection model trained on popular datasets such as INRIA person dataset and Caltech pedestrian dataset containing videos taken in indoor and outdoor environments.
2. Deep neural network based cascaded regression for human anatomical keypoints detection and pose estimation using MPII human multi-person dataset and the COCO 2016 keypoints challenge dataset.
3. Activity classification based on distance between keypoints detected by pose estimation using ensemble of different video datasets such as HMDB, ASLAN for activity recognition
4. Unusual activity detection based on both detection of individual activity or classification of sequence of activities using Recurrent Neural Network LSTM.
Results
The preliminary experimental results for unusual human activity detection model using LSTM recurrent neural network are described below:
Applications
Visual Surveillance through Content Based Retrieval
Security related applications such as Defence, Military, checking in the airports
Home-based rehabilitation
Assisting the sick and disabled patients for Ambient Assisted Living environment
Remote monitoring of elderly persons, jail inmates, patients in hospitals, care-centres for differently abled people
References
1. R. Helaoui, M. Niepert, and H. Stuckenschmidt, “Recognizing interleaved and concurrent activities: A statisticalrelational approach,” in Pervasive Computing and Communications (PerCom), 2011 IEEE International Conference on, pp. 1–9, IEEE, 2011.
2. Aditi Kapoor, K. K. Biswas, M. Hanmandlu, “Unusual Human Activity Detection using Markov Logic Networks, “ presented at Int. Conf. on Identity, Security and Behavior Analysis (ISBA 2017), New Delhi, Feb 24 – 26 , 2017.
3. Zhong, Hua, Jianbo Shi, and Mirkó Visontai. "Detecting unusual activity in video." Computer Vision and Pattern Recognition, 2004. CVPR 2004. Proceedings of the 2004 IEEE Computer Society Conference on. Vol. 2. IEEE, 2004.
4. Zelnik-Manor, Lihi, and Michal Irani. "Event-based analysis of video." Computer Vision and Pattern Recognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on. Vol. 2. IEEE, 2001.
5. Laptev, Ivan, et al. "Learning realistic human actions from movies." Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on. IEEE, 2008.
6. Chaquet, Jose M., Enrique J. Carmona, and Antonio Fernández-Caballero. "A survey of video datasets for human action and activity recognition." Computer Vision and Image Understanding 117.6 (2013): 633-659.